Gesture-based user interface for SmartWalker

In this work, a novel gesture recognition system is presented that allows the control of the SmartWalker by performing simple gestures. The SmartWalker is a normal walker equipped with sensors and actuators that can move autonomously. It is constructed as a walking aid for elderly people. Therefore our gesture recognition system is developed for the same target group which means that gestures with more deviation from the specified gestures need to be supported. We developed two gesture recognition systems. User tracking uses skeleton information extracted from the stream of an RGB+D camera and calculates relative distances and angles between body joints as features. Hand tracking uses the center position of one hand to extract features based on the current hand position relative to a gesture starting point. With the help of Dynamic Time Warping, gestures can be performed at different speeds compared to the training data. For elderly people this is important as persons with physical constrains perform the gestures with irregular movements. To evaluate the performance of our gesture recognition system, we did a performance study where we tested both hand and user tracking at different distances. Additionally, we visited several nursing homes to test our robot and the new gesture recognition system with elderly people. The achieved recognition rates are lower than state-of-the-art gesture recognition systems. Our system on the other hand can handle gestures that are given with large deviations from the specified gestures, it works in unknown environments and supports distances of up to 3.5 meters.